7.1 The Capacity of a Channel

77

Fig. 7.1 Schematic diagram of subprocesses involved in transmitting a signal from a source to a

receiver. Not all of the subprocesses shown are necessary, as discussed in the text. Noise may enter

from the environment or may be intrinsic to the channel hardware

Fig. 7.2 Schematic diagram of the subprocesses involved in carrying out a physical experiment

and transmitting the results

and regards the mouth as the source, the mouth is not the receiver: The sounds are

encoded as patterns of air waves and decoded via tiny mechanical movements in the

ear.

What is the flow of information in the formal scheme of Fig. 7.1? In the previous

chapter, we essentially only considered one agent, who himself carried out an oper-

ation (such as measuring the length of a piece of wood), which reduced uncertainty

and hence resulted in a gain of information according to Eq. (6.7) and further quan-

tified by Eq. (6.5). We now consider that the information is encoded and transmitted

(Fig. 7.2); indeed, it could be broadcast to an unlimited number of people. If they

desired to know the length of that piece of wood and if the structure of their ignorance

was the same as that of the measurer prior to the measurement (i.e., that the wood

was less than a foot long, and they expected to receive the length in inches), then

all those receiving that information would gain the same amount. The transmitted

signals therefore have the potential for making a selection, by operating on the pre-

defined set of alternatives, in exactly the same way as the actual act of measurement

itself. The information content of signals is based on this potential for discrimination.

Hartley, in his pioneering paper, referred to the successive selection of signs from a

given list. This is of course precisely what happens when sending a telegram.

7.1

The Capacity of a Channel

Channel capacity is essentially dependent on the physical form of the channel. If the

channel is constituted by a runner bearing a scroll on which the message is inscribed,

the capacity, in terms of number of messages per day, depends on the distance the